In display remoting technologies, the graphical user interface (GUI) of an application or operating system is captured at a server and sent as a stream of pixel data to a client device at the location of a user. Inputs by the user into the client device may be captured and transmitted to the application or operating system running at the server. Display remoting is used in a variety of circumstances, including when using Internet conferencing applications, when remotely accessing home or office computers, when using Microsoft Windows terminal services, instant gaming (or “game streaming”) services, or when accessing a virtual desktop infrastructure (VDI), among others.
When a user connects to a lightly-loaded remoting server over a fast LAN network, the user experience can be comparable to consuming the content locally, i.e., comparable with using a locally-executing desktop. However, in environments with limited network bandwidth and/or CPU resources, the performance of display remoting technologies may deplete as demands increase, resulting in degraded user experience such as jerky or delayed animation. In certain situations where a user interacts with the GUI of a client device, for example when the user drags a UI object on the screen or scrolls a window, poor remote displaying performance may be particularly impacting on user experience because the interface might appear unresponsive to the user's actions or the interaction may appear jerky. What is needed is a more efficient approach for remote graphical user interface streaming.